{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## We are going to build an Image Classification solution here. The dataset for this is available on Kaggle Link"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import keras"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Import all the Keras libraries here. Keras comes with many layers and we are importing all the layers required here. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.datasets import mnist ## Data set is imported here\n",
    "from keras.models import Sequential # Import the sequential layer. \n",
    "#Generally there are two types of layers, sequential and functional. Sequential is most common one\n",
    "from keras.layers import Dense, Dropout, Flatten\n",
    "from keras.layers import Conv2D, MaxPooling2D\n",
    "from keras import backend as K"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Lets define the hyper parameters here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "batch_size = 256\n",
    "num_classes = 10\n",
    "epochs = 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# input image dimensions\n",
    "image_rows, image_cols = 28, 28"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# here we are diving the dataset in train and test\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "An image has a dimension each for row, height and column. And there are 2 ways to represent\n",
    "If the image format is channels_first it means fist channel is the color [channel][row][col] else\n",
    "if in the case of channels_last it is [row][col][channels]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "if K.image_data_format() == 'channels_first':\n",
    "    x_train = x_train.reshape(x_train.shape[0], 1, image_rows, image_cols)\n",
    "    x_test = x_test.reshape(x_test.shape[0], 1, image_rows, image_cols)\n",
    "    input_shape = (1, image_rows, image_cols)\n",
    "else:\n",
    "    x_train = x_train.reshape(x_train.shape[0], image_rows, image_cols, 1)\n",
    "    x_test = x_test.reshape(x_test.shape[0], image_rows, image_cols, 1)\n",
    "    input_shape = (image_rows, image_cols, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_train shape: (60000, 28, 28, 1)\n",
      "60000 train samples\n",
      "10000 test samples\n"
     ]
    }
   ],
   "source": [
    "x_train = x_train.astype('float32')\n",
    "x_test = x_test.astype('float32')\n",
    "x_train /= 255\n",
    "x_test /= 255\n",
    "print('x_train shape:', x_train.shape)\n",
    "print(x_train.shape[0], 'train samples')\n",
    "print(x_test.shape[0], 'test samples')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert class vectors to binary class matrices\n",
    "y_train = keras.utils.to_categorical(y_train, num_classes)\n",
    "y_test = keras.utils.to_categorical(y_test, num_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Lets create our network here\n",
    "We are adding layers to our network here.\n",
    "Conv2D: 2 dimensional convolutional layer 32: filters required. 3,3: size of the filter (3 rows, 3 columns)\n",
    "Input Image shape is 64*64*3 - height*width*RGB. Each number represents pixel intensity (0-255)\n",
    "Output is a feature map. The training data will work on it and get some feature maps\n",
    "\n",
    "Lets add the activation function now. We are using ReLU (Rectified Linear Unit).\n",
    "The activation function gives the output basis the output. \n",
    "In the feature map output from the previous layer, \n",
    "the activation function will replace all the negative pixels with zero\n",
    "We do not want out network to be overly complex computationally, hence the pooling layer comes into picture\n",
    "The pooling layer will reduce the dimensions. Max with two by two filter, \n",
    "will take the maximum value but the significant features will be retained\n",
    "To fight overfitting using Drop out. Prepare the data by flattening it. And flattening to 1 dimension\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /Users/vaibhavverdhan/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Colocations handled automatically by placer.\n",
      "WARNING:tensorflow:From /Users/vaibhavverdhan/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()\n",
    "model.add(Conv2D(32, kernel_size=(3, 3),\n",
    "                 activation='relu',\n",
    "                 input_shape=input_shape))\n",
    "model.add(Conv2D(64, (3, 3), activation='relu'))\n",
    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "model.add(Dropout(0.45))\n",
    "model.add(Flatten())\n",
    "model.add(Dense(128, activation='relu'))\n",
    "model.add(Dropout(0.55))\n",
    "model.add(Dense(num_classes, activation='softmax'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Compile the model now. We are using crossentropy as the loss function and Adadelta for optimization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss=keras.losses.categorical_crossentropy,\n",
    "              optimizer=keras.optimizers.Adadelta(),\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "The network is ready. Fit the model now"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /Users/vaibhavverdhan/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.cast instead.\n",
      "Train on 60000 samples, validate on 10000 samples\n",
      "Epoch 1/10\n",
      "60000/60000 [==============================] - 55s 916us/step - loss: 0.3784 - acc: 0.8814 - val_loss: 0.0852 - val_acc: 0.9711\n",
      "Epoch 2/10\n",
      "60000/60000 [==============================] - 56s 938us/step - loss: 0.1267 - acc: 0.9625 - val_loss: 0.0512 - val_acc: 0.9818\n",
      "Epoch 3/10\n",
      "60000/60000 [==============================] - 112s 2ms/step - loss: 0.0962 - acc: 0.9720 - val_loss: 0.0393 - val_acc: 0.9873\n",
      "Epoch 4/10\n",
      "60000/60000 [==============================] - 57s 956us/step - loss: 0.0788 - acc: 0.9763 - val_loss: 0.0335 - val_acc: 0.9884\n",
      "Epoch 5/10\n",
      "60000/60000 [==============================] - 57s 951us/step - loss: 0.0698 - acc: 0.9793 - val_loss: 0.0336 - val_acc: 0.9881\n",
      "Epoch 6/10\n",
      "60000/60000 [==============================] - 448s 7ms/step - loss: 0.0622 - acc: 0.9807 - val_loss: 0.0375 - val_acc: 0.9869\n",
      "Epoch 7/10\n",
      "60000/60000 [==============================] - 56s 931us/step - loss: 0.0579 - acc: 0.9823 - val_loss: 0.0286 - val_acc: 0.9903\n",
      "Epoch 8/10\n",
      "60000/60000 [==============================] - 57s 944us/step - loss: 0.0520 - acc: 0.9837 - val_loss: 0.0288 - val_acc: 0.9904\n",
      "Epoch 9/10\n",
      "60000/60000 [==============================] - 2774s 46ms/step - loss: 0.0479 - acc: 0.9854 - val_loss: 0.0307 - val_acc: 0.9898\n",
      "Epoch 10/10\n",
      "60000/60000 [==============================] - 56s 936us/step - loss: 0.0469 - acc: 0.9858 - val_loss: 0.0292 - val_acc: 0.9900\n",
      "Test loss: 0.029244637563071593\n",
      "Test accuracy: 0.99\n"
     ]
    }
   ],
   "source": [
    "model.fit(x_train, y_train,\n",
    "          batch_size=batch_size,\n",
    "          epochs=epochs,\n",
    "          verbose=1,\n",
    "          validation_data=(x_test, y_test))\n",
    "score = model.evaluate(x_test, y_test, verbose=0)\n",
    "print('Test loss:', score[0])\n",
    "print('Test accuracy:', score[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "    "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
